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Garbage in means garbage out. Yes, it’s a cliché, but it’s the best reason I can think of to 
take the time and effort needed to write a good questionnaire for your next market 
research survey. Aside from the interpretation of the results, the questionnaire itself 
(whether it is to be filled out by a respondent or used by an interviewer to ask questions) 
is probably the single most important determinant of how useful the survey results will 
be. And whether you write it yourself or must evaluate or direct the work of a colleague 
or outside consultant, knowing the basics of what constitutes a good questionnaire can 
help make the difference between a study that will collect dust on a shelf and one that 
will illuminate the road ahead and galvanize your plans. Here are a few tips that can help 
make the difference. 


BEFORE WRITING THE QUESTIONNAIRE 

Before you begin, you should ask yourself some planning-oriented questions. What are 
your objectives? Exactly why are you doing the survey? What decisions will you make, 
based on what you learn? The answers to these questions will be the driving force behind 
the questionnaire. Knowing precisely what you want to accomplish will help you 
structure the questionnaire and give you clues about how to tabulate and digest the 
information. If you’re too vague, you’ll probably end up with survey data that won’t be 
focused enough to act on. For example, it may be too general to say, “I want to find out 
what customers think about my product.” A better, more explicit objective may be to find 
out exactly what product changes would be best received by or most useful to customers, 
so that you can increase sales by improving customer satisfaction (or whatever). Or it can 
be to find out what aspects of the product most strongly influence buying decisions— 


which can lead to changes in your advertising messages. 


Also, the more focused your objectives, the better, because a questionnaire can hold a 
respondent’s attention for only about 15 to 30 minutes, depending on the topic, the nature 
of the survey and product, and the quality of the questionnaire. Researchers in the 
computer industry are fortunate in that people seem to enjoy talking about our products; 


thus, they are often more tolerant of longer questionnaires about computer products than 


of those about some other products. However, at some point even computer enthusiasts 


tire and lose interest. Subsequently, the quality of their answers drops rapidly. 


Also, you should determine what kinds of customers you will question and how you will 
group them in your cross-tabulations. Make sure to ask questions that will identify 
respondents by their respective market segments or usage groups, and structure the 
questionnaire and tabulation instructions accordingly. That way, your results will more 
clearly show the different needs of these groups. For example, if you plan to use the 
information to make product improvements, you may find that accountants want 
extended math capabilities whereas users doing marketing jobs want better text 
processing. If you can’t segment the various types of users, your research will show only 
that some users want this and some want that. The “who” and “why” will remain a 


mystery. 


All these things—objectives, plans for using the information, whom to survey—should 
be committed to paper. Everyone involved in the project, especially those who will write 
the questionnaire, conduct the survey, and interpret the results, should fully understand 


them. 


STRUCTURING THE QUESTIONNAIRE 

The order in which you ask questions can be almost as important as the questions 
themselves. Although questionnaires run the gamut—depending on objectives, timing, 
and budget—a good one usually flows like this: 

1. An introductory script or cover letter that identifies the interviewer and topic. 

2. Questions to qualify respondents. 

3. General questions about the main issue/topic. 

4. Specific questions about the main issue/topic. 

5. Questions about secondary issues. 


6. Questions to determine respondent demographics or characteristics. 


Introduce Yourself. This should be very brief and to the point. The script for a phone 
survey might say, “Good afternoon. This is Fred Smith of Alpha Omega Software. My 
company is surveying users of our XYZ product. Our registration records show that you 


recently purchased this product. Do you have a few minutes to answer some questions?” 


If so, go immediately to the first question. If not, ask if there is a more convenient time to 
call back. If you’re doing a mail survey, the cover letter or beginning/introduction to the 
survey should be similarly brief; be sure to include a statement of your gratitude for the 


respondent’s taking the time to fill it out and send it back. 


Qualify the Respondent. Do this early in the survey to make sure you’re talking to the 
right people. Ask very specific questions that will help you determine that these people 
have the appropriate background to give you credible answers. For example: “Do you 
own product XYZ? How often do you use it: every day, a few times a week, a few times 
a month, or less often? Do you use it in business, for personal enjoyment, or for some 
other reason?” If the respondents don’t qualify (that is, meet your criteria), tell them 


you’ re finished and thank them for their time. 


If you’re doing a mail survey, the qualification should take place either before you mail 
the survey or in the cover letter. In the letter, you might say, “Your name is on our list of 
registered owners of XYZ product. If you aren’t the person who uses it most, please give 


this questionnaire to the person in your department who does use it most.” 


Move from the General to the Specific. Once you know you’ve contacted the right 
person, you can begin to broach the survey topic. Start with basic background questions 
about your issue or subject that begin to bring it into focus for the respondent. Then get 
more specific. Some general questions might be, “(How long have you used product 
XYZ? Which version are you using? Have you used a previous version? Have you used a 


competing product? What is your main purpose in using XYZ?” 


With the general questions answered, you can then ask more specific, probing ones. 
“What do you especially like about XYZ? What else?” If you are fishing for something 
particular in a phone interview and the respondent doesn’t mention it, you can ask about 
it specifically: “How would you rate XYZ on its ability to handle multiple data sets? 
What do you dislike about product XYZ?” You might also want to encourage the 


respondent to be frank with you and indicate that an honest response won’t offend you. 


These “likes and dislikes” questions are usually open-ended; they have no set of precoded 
responses for the interviewer to check off. However, if you can conjecture about likely 


answers, you can save a lot of interviewing (and data-processing) time by writing the 


probable answers into the questionnaire. But be sure to leave plenty of space for 


responses in the ““other—please specify” category. 


Explore Secondary Issues. At this point, if you have space or time, you may want to 
probe more deeply to elicit responses about your primary issues or to ask about secondary 
ones. Secondary issues are usually those that ended up taking a backseat when you finally 
came up with the study’s objectives. For example, you may have decided that the study 
would primarily address product changes; pricing and packaging were also important, but 


less so for this study. 


Record Demographics, Personal Characteristics. You can conclude the survey with 
questions that people are more comfortable and familiar with, such as those about 
demographics or personal characteristics. For example: “Do you own a home or rent? Do 
you have children? If so, how many and what ages? What is your job title, industry, and 
the size of your company? What is your age, education level, income?” Be aware that 
questions about income can put off many people. You may opt not to ask about it, or you 
can ask for an income range, such as, “Is your income $10,000 to $20,000? $20,000 to 
$30,000?” and so forth. 


To close, you may want to ask if the respondent wishes to say anything more about the 
XYZ product or the company that makes it. And always end the survey, whether by 


phone or mail, with a polite “Thank you very much for your time.” 


CHOOSING THE RIGHT STYLE OF QUESTION 

There are several styles of questions to choose from. Whichever you use should be 
dictated by your needs. If you want to know “why” or “what,” you should ask open- 
ended questions that allow for unstructured answers. If you want to know “how many,” 


ask closed-end questions that restrict the respondent to a list of possible answers. 


Surveys with mostly closed-end questions are much faster for a respondent to complete 
and are much less expensive; they reduce interviewing time and data-processing efforts. 
The more closed-end questions you ask and the more specific they are, the quicker and 
cheaper the survey process. However, you should always allow time/space for an “other” 
category. Many people will have answers outside of those you’ve provided, so if you give 
them an opportunity to specify “other,” it makes them feel better about the survey—and 


you get better information as well. 


When and How to Use Scales. Most questions require more than a simple yes or no 
response. When you want respondents to rate or compare things or answer how much, 
how often, and so forth, a good way to structure those answers is to use a scale or 
ranking. Devise one, and ask the respondents to choose the option that suits them. For 
example, if you ask how often someone uses a product, potential rankings might include 
never, not very much, sometimes, a lot, once a day, twice a day, twice a week, rarely, 
often, three hours a day, three hours a week, and so forth. To get usable and 
understandable data, devise a scale that is realistic, that means something to a marketing 


manager, and that doesn’t allow ambiguous answers. 


If you want to rate and compare things, a 1-to-10 scale can be useful. Usually, a ranking 
of 1 means poor and 10 means excellent. This makes it easy to compare average scores. 
Be careful, though. Be sure to look at how the answers are distributed over the scale; you 
may get an average of 5, but if you don’t examine the distribution, you won’t know that 


half the people rated the item a 1 and half gave it a 10. 


“WHO SPONSORED THIS STUDY?” 

Whether to identify the survey’s sponsor is a real judgment call. It’s usually better to 
leave the sponsor’s name out of the process. That way, any biases people have toward the 
sponsor will not be reflected in the results. However, if you are investigating a specific 
product or trying to gauge the perception of a company’s image, it may be best to reveal 
the sponsor. That’s OK, as long as you screen respondents to make sure they aren’t 


working for a competitor or a competitor’s advertising or PR agency. 


Often you can ask several questions up front before having to reveal the sponsor. 
However, sometimes it is helpful to reveal the sponsor at the start so that the respondent 
perceives that the survey is legitimate (this is often the case with business people or in 
very product-specific surveys). Sometimes it is possible to complete a questionnaire 
without revealing the sponsor. In such cases, if you haven’t given out any confidential 
product information during the survey, you may want to give the company a little PR 
boost and close by telling the respondent that “This was sponsored by the XYZ company; 
they appreciate your time and thoughtfulness. Do you have anything else you’d like to 


say to them before we conclude?” 


PILOT-TEST THE QUESTIONNAIRE 


Before you do the survey on a large scale, it’s always a good idea to test a questionnaire 
to make sure that the questions flow well and that respondents clearly understand what 
you are asking. A good pilot involves going through all the actual steps—using a much 
smaller sample—that you would take if you were doing the full-blown survey. Then you 
can check with the interviewers and respondents to see if there was any confusion or to 
identify places in which the flow was rough. With phone surveys, it’s a good idea to 


listen in on at least a couple of interviews. 


A shortcut mail survey pilot involves administering it to some people in your office or 


their spouses and getting their feedback. 


So ask the right people the right questions and you will get the right kinds of answers— 


those you can act on. By avoiding the “garbage out” syndrome, you’ll save time and 


money and will more likely start your decision-making process on the right foot. 
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